[WIP] Add umbrella optimization flags #20047

IanWood1 · 2025-02-20T20:59:33Z

This change adds support for optimization flags that can be used to broadly control compiler optimizations. Currently, iree-opt-level is the name of the flag that would control the overall optimization level for the compiler invocation and it would cascade down to more fine-grained flags (i.e. iree-opt-strip-assertions). This will give users an easy way to control the compilation process and reduce the number of flags required to get the desired optimizations.

Progress towards #19072

ScottTodd · 2025-02-20T22:02:38Z

compiler/src/iree/compiler/Pipelines/Options.cpp

+void GlobalPipelineOptions::bindOptions(OptionsBinder &binder) {
+  static llvm::cl::OptionCategory category(
+      "IREE global pipeline options controlling the entire compilation flow.");
+
+  binder.opt<llvm::OptimizationLevel>(
+      "iree-opt-level", optLevel,
+      llvm::cl::desc("Global optimization level to apply to the entire "
+                     "compilation flow."),
+      llvm::cl::cat(category));
+}


Just dropping a reminder here: we have some existing docs for "optimization options" that are due to a rework, especially after these changes land: https://iree.dev/reference/optimization-options/

We could even add a snippet that we include on each page like https://iree.dev/guides/deployment-configurations/cpu/#compile-and-run-a-program and https://iree.dev/guides/deployment-configurations/gpu-vulkan/#compile-and-run-a-program that explains what to expect out of the box and how to optimize / tune / configure from there (linking to the optimization options reference page).

IanWood1 · 2025-02-20T23:51:18Z

compiler/src/iree/compiler/Pipelines/Options.cpp

+void GlobalOptimizationOptions::applyOptimization(
+    const OptionsBinder &binder, const GlobalPipelineOptions &globalLevel) {
+  binder.overrideDefault("iree-global-optimization-opt-level", optLevel,
+                         globalLevel.optLevel);
+
+  if (optLevel != llvm::OptimizationLevel::O0) {
+    binder.overrideDefault("iree-opt-strip-assertions", stripAssertions, true);
+    llvm::dbgs() << "stripAssertions " << stripAssertions << '\n';
+  }
+};
+
 void GlobalOptimizationOptions::bindOptions(OptionsBinder &binder) {
  static llvm::cl::OptionCategory category(
      "IREE options for controlling global optimizations.");
+  binder.opt<llvm::OptimizationLevel>(
+      "iree-global-optimization-opt-level", optLevel,
+      llvm::cl::desc("Optimization level for the this pipeline"),
+      llvm::cl::cat(category));


@benvanik I attempted to do similar to what you suggested but I've run into some problems.

As an example init_at_opt_level would be the function used to tell the binder to set stripAssertions to true when at O3 (and there wouldn't be a need for the manual applyOptimization):

binder.opt<bool>( "iree-opt-strip-assertions", stripAssertions, init_at_opt_level(O3, true));

The problem I'm having is that iree-opt-strip-assertions should be set based upon the value iree-global-optimization-opt-level and I don't have a good way to let the binder know that.

There is a secondary problem, too. As you mentioned, the options shouldn't be directly set but rather a copy should be made. This is needed because we don't want setting optimization levels to affect the other flags for a session. This makes it hard (impossible?) to to make it so that GlobalOptimizationOptions::stripAssertions should be set based on GlobalOptimizationOptions::optLevel from inside of of bindOptions because the GlobalOptimizationOptions object we use to bind won't be the one that has the optimizations applied.

This is a real good start!

There's a few super magic ways we could do things, but for v0 I'd have your applyOptimization above walk through the options registered on a binder and if there are registered init_at_opt_level entries perform the override. It's a bit of boilerplate we could remove with template magic but copy/pasting for now is fine. something like:

applyOptimization: binder.overrideDefault("...-opt-level", optLevel, ...); binder.overrideOptimizationDefaults(optLevel); // if there are nested binders, call applyOptimization on them

The fancier automatic way would be to have the pipeline opt level option be stashed by the binder - instead of binder.opt<OptimizationLevel>(...) could have binder.optimizationLevel("...-opt-level") to make it explicit what the primary scoped optimization level option is and then the applyOptimization default impl above can do everything itself: the top level gets called, it checks if it has a registered optimization level option, overrides it if needed, and then calls overrideOptimizationDefaults. Then we'd only need to override it if we had nested options (but even that could be improved, and for now we could ignore).

- Use llvm::OptimizationLevel - Fix `isChanged` callback and move previus impl to `isDefault` - Add tests for OptionUtils Signed-off-by: Ian Wood <[email protected]>

Signed-off-by: Ian Wood <[email protected]>

- Uses the overall optimization level to set global opt's phase optimization level. Also, sets strip assertions accordingly - Adds a test to make sure that the session flags are not modified. Signed-off-by: Ian Wood <[email protected]>

Signed-off-by: Ian Wood <[email protected]>

IanWood1 · 2025-02-21T21:19:35Z

This is still a WIP, but I opened the PR and requested reviews to get some feedback on the general approach. There is definitely some cleanup needed + only 1 flag is setup to use the optimization levels. I would like to hear some thoughts on #20047 (comment) before flushing this out.

benvanik · 2025-02-24T17:56:26Z

compiler/src/iree/compiler/Pipelines/Options.cpp

+void GlobalOptimizationOptions::applyOptimization(
+    const OptionsBinder &binder, const GlobalPipelineOptions &globalLevel) {
+  binder.overrideDefault("iree-global-optimization-opt-level", optLevel,
+                         globalLevel.optLevel);
+
+  if (optLevel != llvm::OptimizationLevel::O0) {
+    binder.overrideDefault("iree-opt-strip-assertions", stripAssertions, true);
+    llvm::dbgs() << "stripAssertions " << stripAssertions << '\n';
+  }
+};
+
 void GlobalOptimizationOptions::bindOptions(OptionsBinder &binder) {
  static llvm::cl::OptionCategory category(
      "IREE options for controlling global optimizations.");
+  binder.opt<llvm::OptimizationLevel>(
+      "iree-global-optimization-opt-level", optLevel,
+      llvm::cl::desc("Optimization level for the this pipeline"),
+      llvm::cl::cat(category));


This is a real good start!

There's a few super magic ways we could do things, but for v0 I'd have your applyOptimization above walk through the options registered on a binder and if there are registered init_at_opt_level entries perform the override. It's a bit of boilerplate we could remove with template magic but copy/pasting for now is fine. something like:

applyOptimization: binder.overrideDefault("...-opt-level", optLevel, ...); binder.overrideOptimizationDefaults(optLevel); // if there are nested binders, call applyOptimization on them

The fancier automatic way would be to have the pipeline opt level option be stashed by the binder - instead of binder.opt<OptimizationLevel>(...) could have binder.optimizationLevel("...-opt-level") to make it explicit what the primary scoped optimization level option is and then the applyOptimization default impl above can do everything itself: the top level gets called, it checks if it has a registered optimization level option, overrides it if needed, and then calls overrideOptimizationDefaults. Then we'd only need to override it if we had nested options (but even that could be improved, and for now we could ignore).

IanWood1 · 2025-02-25T18:49:14Z

@benvanik replying to your comment above, I was trying to get this to work but I think there is a problem with having the binder automatically apply flag modifications specified with init_at_opt_level.

Maybe its a problem with how I'm trying to implement it, so I will share some details. I was thinking that init_at_opt_level would basically instruct the binder to store a callback that would take store reference to the member of the options struct (e.g. stripAssertions). That way it can get updated in applyOptimization. However, this is dangerous because the reference stored in the binder is not always the same object that applyOptimizations is called on. This is why I initially opted for manual updates in applyOptmizations.

For example:

GlobalOptimizationOptions options0;
{
  GlobalOptimizationOptions options1;
  // binds options and stores reference to `options1.stripAssertions` to be updated 
  // when `applyOptimizations` is called.
  options1.bindOptions(binder);
  options0 = optioins1;
}

// set from flags...

// Problematic because binder is storing references to options1's members.
// So, `options0` never gets updated and this is UB
options0.applyOptimizations(binder);

benvanik · 2025-02-25T18:51:58Z

I didn't think of using callbacks and instead just storing the data as a field on the options info that is cloneable - SmallVector or something of the optimization level -> value.

IanWood1 · 2025-02-25T19:10:07Z

I didn't think of using callbacks and instead just storing the data as a field on the options info that is cloneable - SmallVector or something of the optimization level -> value.

Okay, so this would still require something that passes this field to the binder? Something roughly similar to

void applyOptimizations(...){
  binder.overrideDefault("...-opt-level", optLevel, ...);
  binder.overrideDefaultWith(optLevel, stripAssertions, stripAssertionsOptToDefaultMap);
}

And the benefit would be printability of default levels at each optimization level because they are declared during binding?

benvanik · 2025-02-25T19:34:19Z

I don't think so. You'd have a subclass of opt (or Option) - optimization_level_opt or something - that added an std::vector<std::tuple<OptimizationLevel, DataType>> of your map and an overload of setInitialValue that took an optimization level along with the value and added to that vector. Then you would have your element declaration struct init_at_opt_level (or whatever) ala init/initializer that calls your overloaded setInitialValue function. Your custom printOptionInfo could then walk the vector and print out the values. OptionsBinder::opt could then be changed to do a make_unique of your new opt type instead of the base one and you could stash in the LocalOptionInfo that it was something to apply optimizations to (I don't think there's dyn_cast unfortunately). Your top-level applyOptimizationLevel would walk through the LocalOptionInfo and if the option was your custom opt type cast and do whatever it needs with the std::vector on it.

A user should then be able to do something like

 binder.opt<bool>(
      "iree-opt-strip-assertions",
      stripAssertions,
      init_at_opt_level(O2, true));

(that make_unique'ing your optimization_level_opt, the init_at_opt_level calling the setInitialValue(O3, true) to add it to the opt vector, and your generic applyOptimizations walking over it to do the "if initial value then walk vector and find the max opt level value and use that" (e.g. the O2 option should be picked up above if -O3 was specified))

(rough idea, but it should work just as list works today with cloning and such for local scopes, still appear as an opt to most users and take all the same declarations, just also takes the new ones)

Signed-off-by: Ian Wood <[email protected]>

IanWood1 · 2025-02-27T22:03:47Z

experimental/regression_suite/shark-test-suite-models/sdxl/test_unet.py

-    "--iree-opt-strip-assertions=true",
    "--iree-global-opt-propagate-transposes=true",
    "--iree-dispatch-creation-enable-fuse-horizontal-contractions=true",
    "--iree-dispatch-creation-enable-aggressive-fusion=true",
-    "--iree-opt-aggressively-propagate-transposes=true",
-    "--iree-opt-outer-dim-concat=true",
-    "--iree-opt-generalize-matmul=true",


Picked a few (hopefully uncontroversial flags) to move under the optimization defaults

Signed-off-by: Ian Wood <[email protected]>

ScottTodd reviewed Feb 20, 2025

View reviewed changes

IanWood1 commented Feb 20, 2025

View reviewed changes

IanWood1 added 4 commits February 21, 2025 00:30

Add functionality and tests

e25afda

- Use llvm::OptimizationLevel - Fix `isChanged` callback and move previus impl to `isDefault` - Add tests for OptionUtils Signed-off-by: Ian Wood <[email protected]>

Fix tests and add another

a9147ad

Signed-off-by: Ian Wood <[email protected]>

Use optimization for global optimization phase

11d6258

- Uses the overall optimization level to set global opt's phase optimization level. Also, sets strip assertions accordingly - Adds a test to make sure that the session flags are not modified. Signed-off-by: Ian Wood <[email protected]>

Add basic iree-compile test

882ee69

Signed-off-by: Ian Wood <[email protected]>

IanWood1 changed the title ~~[WIP] Add optimization flags~~ [WIP] Add umbrella optimization flags Feb 21, 2025

IanWood1 marked this pull request as ready for review February 21, 2025 21:15

IanWood1 requested a review from benvanik as a code owner February 21, 2025 21:15

IanWood1 requested review from MaheshRavishankar and qedawkins February 21, 2025 21:15

benvanik requested changes Feb 24, 2025

View reviewed changes

IanWood1 mentioned this pull request Feb 25, 2025

New flag: iree-codegen-math-transform-tweaks #20042

Open

IanWood1 force-pushed the opt_level branch 2 times, most recently from 46d1505 to 1f8ae49 Compare February 27, 2025 06:00

IanWood1 added 3 commits February 27, 2025 09:56

Add init_at_opt

1f8ae49

Signed-off-by: Ian Wood <[email protected]>

Fix template and clean up

a9c362b

Signed-off-by: Ian Wood <[email protected]>

Fix asan

1a96192

Signed-off-by: Ian Wood <[email protected]>

IanWood1 requested a review from stellaraccident as a code owner February 27, 2025 22:02

IanWood1 commented Feb 27, 2025

View reviewed changes

IanWood1 requested a review from benvanik February 27, 2025 22:07

IanWood1 added 2 commits February 28, 2025 01:38

Print defaults at each optimization level & add test

0b97479

Signed-off-by: Ian Wood <[email protected]>

Implement a few other straightforward unet flags

fb4e64f

Signed-off-by: Ian Wood <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Add umbrella optimization flags #20047

[WIP] Add umbrella optimization flags #20047

IanWood1 commented Feb 20, 2025 •

edited

Loading

ScottTodd Feb 20, 2025

IanWood1 Feb 20, 2025

benvanik Feb 24, 2025

IanWood1 commented Feb 21, 2025

benvanik Feb 24, 2025

IanWood1 commented Feb 25, 2025 •

edited

Loading

benvanik commented Feb 25, 2025

IanWood1 commented Feb 25, 2025 •

edited

Loading

benvanik commented Feb 25, 2025

IanWood1 Feb 27, 2025 •

edited

Loading

[WIP] Add umbrella optimization flags #20047

Are you sure you want to change the base?

[WIP] Add umbrella optimization flags #20047

Conversation

IanWood1 commented Feb 20, 2025 • edited Loading

ScottTodd Feb 20, 2025

Choose a reason for hiding this comment

IanWood1 Feb 20, 2025

Choose a reason for hiding this comment

benvanik Feb 24, 2025

Choose a reason for hiding this comment

IanWood1 commented Feb 21, 2025

benvanik Feb 24, 2025

Choose a reason for hiding this comment

IanWood1 commented Feb 25, 2025 • edited Loading

benvanik commented Feb 25, 2025

IanWood1 commented Feb 25, 2025 • edited Loading

benvanik commented Feb 25, 2025

IanWood1 Feb 27, 2025 • edited Loading

Choose a reason for hiding this comment

IanWood1 commented Feb 20, 2025 •

edited

Loading

IanWood1 commented Feb 25, 2025 •

edited

Loading

IanWood1 commented Feb 25, 2025 •

edited

Loading

IanWood1 Feb 27, 2025 •

edited

Loading